Autonomous vehicles (AVs) require continuous sensor data processing in order to operate through road traffic on public roads in order to match or even surpass human capabilities. AVs can be equipped with many kinds of sensors, including stereoscopic cameras and optical flow sensors, but processing images from a stereoscopic camera in real-time with enough fidelity to properly identify and classify obstacles is a challenge. Moreover, adding additional sensors requires more processing power and generates redundant and extraneous data.
In stereo vision, images are captured from a pair of cameras or lenses of a camera that are slightly displaced relative to each other. This positional difference is known as horizontal disparity and allows a stereo camera to perceive and calculate depth, or the distance from the camera to objects in a scene. At present, stereoscopic imaging is mostly fulfilled by utilizing a parallax effect. By providing a left image for a left eye and a right image for a right eye, it is possible to convey a 3D impression to a viewer when the viewer is watching the images at an appropriate viewing angle. A two-view stereoscopic video is a video generated by utilizing such an effect and each frame of the video includes an image for a left eye and another image for a right eye. The depth information of objects in the frame can be obtained by processing the two-view stereoscopic video. The depth information for all pixels of the image makes up a disparity map.
Optical flow is the pattern of apparent motion of objects, surfaces, and edges in a visual scene caused by the relative motion between an observer (an eye or a camera) and the scene. The optical flow methods try to calculate the motion, for each pixel or voxel position, between two image frames which are taken at separate times. An optical flow sensor is a vision sensor capable of measuring optical flow or visual motion and outputting a measurement based on optical flow.